Search for Rare and Forbidden Charm Meson Decays D° — ► V£ + £ and hhlt 
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We report results of a search for flavor-changing neutral current (FCNC), lepton- flavor, and 
lepton-number violating decays of the D° (and its antiparticle) into 3 and 4-bodies. Using data 
from Fermilab charm hadroproduction experiment E79f , we examine modes with two leptons (muons 
or electrons) and either a p°, K , or <f> vector meson or a non-resonant tvty, Kty, or KK pair of 
pseudoscalar mesons. No evidence for any of these decays is found. Therefore, we present branching- 
fraction upper limits at 90% confidence level for the 27 decay modes examined (18 new). 

PACS numbers: 11.30.Fs, 12.15.Mm, 13.20.Fc, 14.80.Cp 



The E791 Collaboration has previously reported lim- 
its on rare and forbidden dilepton decays of the charged 
charm D meson Such measurements probe the 

SU(2)xU(l) Standard Model of electroweak interactions 
in search of new mediators and couplings HQ- We ex- 
tend the methodology to 27 dilepton decay modes of the 
neutral D meson. The modes are resonant D° -> V£ + £~ 
decays, where V is a p , K , or <f>, and non-resonant 
D° — > hhtt decays, where h is a ir or K. The lep- 
tons are either muons or electrons. Charge-conjugate 
modes are implied. The modes are lepton flavor- violating 
(e.g., D° — > p°p + e~), or lepton number-violating (e.g., 
— > 7r 7r /i + /i + ), or flavor-changing neutral current 

decays (e.g., D° — ► K e + e~). Box diagrams can mimic 
FCNC decays, but only at the KT 10 to 10" 9 level @,|. 
Long range effects (e.g., D° — > K p°, p° — ► e + e~) 
can occur at the 10 -6 level Numerous experiments 

have studied rare decays of charge -1/3 strange quarks. 
Charge 2/3 charm quarks are interesting because they 
might couple differently 0. 



The data come from measurements with the Fermi- 
lab E791 spectrometer [g]. A total of 2 x 10 10 events 
were taken with a loose transverse energy requirement. 
These events were produced by a 500 GeV/c 7r~ beam 
interacting in a fixed target consisting of five thin, well- 
separated foils. Track and vertex information came from 
"hits" in 23 silicon microstrip planes and 45 wire cham- 
ber planes. This information and the bending provided 
by two dipole magnets were used for momentum analysis 
of charged particles. Kaon identification was carried out 
by two multi-cell Cerenkov counters that provided n/K 
separation in the momentum range 6 — 60 GeV/c [pj. 
We required that the momentum-dependent light yield in 
the Cerenkov counters be consistent for kaon-candidatc 
tracks, except for those in decays with <j> — ► K + K~, 
where the narrow mass window for the <f> decay provided 
sufficient kaon identification (ID). 

Electron ID was based on transverse shower shape plus 
matching wire chamber tracks to shower positions and 
energies in an electromagnetic calorimeter ]l0|] . The elec- 
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tron ID efficiency varied from 62% below 9 GeV/c to 45% 
above 20 GeV/c. The probability to misidentify a pion as 
an electron was ~ 0.8%, independent of pion momentum. 

Muon ID was obtained from two planes of scintillation 
counters. The first plane (5.5 m x 3.0 m) of 15 counters 
measured the horizontal position while the second plane 
(3.0 m x 2.2 m) of 16 counters measured the vertical po- 
sition. There were about 15 interaction lengths of shield- 
ing upstream of the counters to filter out hadrons. Data 
from D + — > K decays |ll|] were used to choose 

selection criteria for muon candidates. Timing informa- 
tion from the smaller set of muon scintillation counters 
was used to improve the horizontal position resolution. 
Counter efficiencies, measured using muons originating 
from the primary target, were found to be (99 ± 1)% for 
the smaller counters and (69 ±3)% for the larger coun- 
ters. The probability for misidentifying a pion as a muon 
decreased with momentum, from about 6% at 8 GeV/c 
to (1.3 ± 0.1)% above 20 GeV/c. 

Events with evidence of well-separated production 
(primary) and decay (secondary) vertices were selected 
to separate charm candidates from background. Sec- 
ondary vertices were required to be separated from the 
primary vertex by greater than 12<r L , where a L is the 
calculated resolution of the measured longitudinal sep- 
aration. Also, the secondary vertex had to be sepa- 
rated from the closest material in the target foils by 
greater than 5 a' L , where a' L is the uncertainty in this 
separation. The vector sum of the momenta from sec- 
ondary vertex tracks was required to pass within 40 pm 
of the primary vertex in the plane perpendicular to the 
beam. Finally, the net momentum of the charm can- 
didate transverse to the line connecting the production 
and decay vertices had to be less than 300 MeV/c. De- 
cay track candidates were required to pass approximately 
10 times closer to the secondary vertex than to the pri- 
mary vertex. These selection criteria and kaon identifi- 
cation requirements were the same for both the search 
mode and for its normalization signal (discussed below). 
The mass ranges used for the resonant masses were: 
Itott+jt- — rripo J < 150 MeV/c 2 , — m-^*o| < 55 

MeV/c 2 , and \m K+K - - m | < 10 MeV/c 2 . 

To determine our selection cuts we used a "blind" anal- 
ysis technique. Before the selection criteria were final- 
ized, all events having masses within a window AMs 
around the mass of the D° were "masked" so that the 
presence or absence of any potential signal candidates 
would not bias our choice of selection criteria. All crite- 
ria were then chosen by studying events generated by a 
Monte Carlo (MC) simulation program and background 
events, outside the signal windows, from real data. The 
criteria were chosen to maximize the ratio Nmc I \N~B, 
where Nmc an d Nb are the numbers of MC and back- 
ground events, respectively, after all selection criteria 
were applied. The data within the signal windows were 



unmasked only after this optimization. We used asym- 
metric windows for the decay modes containing electrons 
to allow for the bremsstrahlung low-energy tail. The sig- 
nal windows were: 1.83 < M(D°) < 1.90 GeV/c 2 for pp 
and 1.76 < M(D°) < 1.90 GeV/c 2 for ee and pe modes. 

We normalize the sensitivity of our search to topo- 
logically similar hadronic 3-body (resonant) or 4-body 
(non-resonant) decays. One exception to this is the case 
of D° -> p°£ ± P where we normalize to nonresonant 
D — > 7T + 7r~7r + 7r~ because there is no published branch- 
ing fraction for D° — > p ir + Ti~ . Table | lists the normal- 
ization mode used for each signal mode and the fitted 
number of data events (A^orm)- 

The upper limit for each branching fraction Bx is cal- 
culated using the following formulae: 



B x = 



Nx £no 



A Nc 



x B 
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-X 



£x 



fMC 
J Norm 

fMC • 

JX 



(1) 



Nx is the 90% confidence level (CL) upper limit on 
the number of decays for the rare or forbidden decay 
mode X and i?Norm is the normalization mode branch- 
ing fraction obtained from the Particle Data Group 
fll2[ . ENorm and ex are the detection efficiencies while 
ftfoiLn ano - fx~ C are the fractions of Monte Carlo events 
that are reconstructed and pass the final selection crite- 
ria, for the normalization and decay modes respectively. 

TABLE I. Normalization modes used. 



Decay Mode 


Normalization Mode 


A^Norm 


D° ^ p ^^ 


D° -> 7T + 7r"7T + 7r" 


2049±53 


d° ^irv^ 


D°— jTVNt- 


5451±72 


D° -> ${±1* 


D° -> (pTT+ir- 


113±19 


D° -> nnll 


D° —* 7T + 7r~7r + 7r - 


2049±53 


D° -> Kn££ 


D° -» K-n+n-n+ 


11550±113 


D° -» KKii 


D° K+K-it+tt- 


406±41 



The MC simulations use Pythia/ Jetset |13| as the 
physics generator and model the effects of resolution, de- 
tector geometry, magnetic fields, multiple scattering, in- 
teractions in the detector material, detector efficiencies, 
and the analysis selection criteria. The efficiencies for the 
normalization modes varied from approximately 0.2% to 
1% depending on the mode, and the efficiencies for the 
search modes varied from approximately 0.05% to 0.34%. 
We take muon and electron ID efficiencies from data. 

Monte Carlo studies show that the experiment's accep- 
tances are nearly uniform across the Dalitz plots, except 
that the dilepton identification efficiencies typically drop 
to near zero at the dilepton mass threshold. While the 
loss in efficiency varies channel by channel, the efficiency 
typically reaches its full value at masses only a few hun- 
dred MeV/c 2 above the dilepton mass threshold. We use 
a constant weak-decay matrix element when calculating 
the overall detection efficiencies. 
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The 90% CL upper limits Nx are calculated using the 
method of Feldman and Cousins [H to account for back- 
ground, and then corrected for systematic errors by the 
method of Cousins and Highland In these methods, 
the numbers of signal events are determined by simple 
counting, not by a fit. All results are shown in Fig. |l| 
and listed in Table Upper limits are determined using 
the number of candidate events observed and expected 
number of background events within the signal region. 
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FIG. 1. Final event samples for the opposite signed dilep- 
ton (rows 1-3) , resonant (rows 4-6) , and same signed dilepton 
modes (rows 7-9) of D° decays. The solid curves display total 
estimated background; the dotted curves display signal shape 
for a number of events equal to the 90% CL upper limit. The 
dashed vertical lines are the AMs boundaries. 



Background sources that are not removed by the se- 
lection criteria discussed earlier include decays in which 
hadrons (from real, fully-hadronic decay vertices) are 
misidentified as leptons. These misidentified leptons can 



come from hadronic showers reaching muon counters, 
decays-in-flight, and random overlaps of tracks from oth- 
erwise separate decays ( "accidental" sources) . In the case 
where kaons are misidentified as pions or leptons, candi- 
date masses shift below signal windows. However, we re- 
move these events to prevent them from influencing our 
background estimate, which is partially obtained from 
the mass sidebands (see discussion of iVcmb below). To 
remove these events prior to the selection-criteria opti- 
mization, we reconstruct all candidates as each of the 
non-resonant normalization modes and test whether the 
masses are consistent with m^a. If so, we remove the 
events, but only if the number of kaons in the final state 
differs from that of the search mode. We do not remove 
events having the same number of kaons, as the loss in 
acceptance for true signal events would be excessive. 

There remain two sources of background: hadronic de- 
cays where pions are misidentified as leptons (-/VmisId) 
and "combinatoric" background (iVcmb) arising primar- 
ily from false vertices and partially reconstructed charm 
decays. The background iVMisiD arises from the nor- 
malization modes. To estimate the rate for misiden- 
tifying tttt as ££, for all but the D° -> K-<K+l + t~ 
modes, we assume all D° ->■ K-Tr + e + £- candidates ob- 
served (after subtracting combinatoric background esti- 
mated from mass sidebands) result from misidentifica- 
tion of D — > K~ir + Tr~ir + decays and count the num- 
ber of D° — > K~ir + £ + £~ decays passing the final se- 
lection criteria. We then divide by twice the num- 
ber of D° — > K~ir + ir~7r + normalization events with 
K ~ir + £ + £~ mass within AMs boundaries (twice because 
there are two possible 7r + misidentifications). 

From this procedure, the following misidentification 
rates were obtained: r MAI = (3.4 ± 2.4) x 10~ 4 , r Me = 
(4.2 ± 1.4) x 1(T 4 , and r ee = (9.0 ± 6.2) x 10~ 5 . For 
modes in which two possible pion combinations can con- 
tribute, e.g., D° -> K~ 'it + (j, fJ^ , we use twice the above 
rate; and for D° — > 7r + 7r~7r + 7r~, where there are 4 possi- 
ble combinations, we use 4 times this rate in calculating 
D° — > ir + ir~£ + £~ . Using these rates, we estimate the 
numbers of misidentified candidates 
in the signal windows as follows: 



A[VU „ j N hhU 



N- 



MisID 

hh-n 



x "Norm dnaJV MisID 



x^No™, (2) 



where -A^orm and iV^m are the numbers of normaliza- 
tion hadronic decay candidates in the signal windows. 

To calculate the upper limits for the D° — > K~ir + £ + £~ 
modes, we set -/VmisID to zero as we do not have an in- 
dependent estimate of the misidentification rates. This 
results in conservative upper limits. If we had used the 
misidentification rates from our previous, 3-body decay 
study then our limits for the three D° — > K~tt + £ + £~ 
modes would be lower by about a factor of two. 

To estimate the combinatoric background ivcmb 
within a signal window AMs, we count events hav- 
ing masses within an adjacent background mass win- 
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dow AMb, and scale this number (Nam b ) by the rel- 
ative sizes of these windows: -ZVcmb = (AMg/AMg) x 
Nam b - To be conservative in calculating our 90% 
confidence level upper limits, we take combinatoric 
backgrounds to be zero when no events are lo- 
cated above the mass windows. Table [n] shows the 
numbers of combinatoric background, misidentification 
background, and observed events for all 27 modes. 

TABLE II. E791 90% confidence level (CL) upper limits 
on the number of events and branching fraction limits (xlO -5 ). 
Previously published limits Jl2|Jl5| for the nine D° — > V£ + £~ 
modes are 23, 10, 4.9, 118, 14, 10, 41, 5.2, and 3.4 x 10~ 5 . 



Mode 


(Est. BG) 




Sys. 




E791 


D° -> 


Afcmb 


A'MisID 


iVobs 


Err. 


N x 


Limit 




0.00 


3.16 


2 


11% 


2.96 


3.0 


-K + -K~e + e~ 


0.00 


0.73 


9 


12% 


15.2 


37.3 


ty + ir~ 


5.25 


3.46 


1 


15% 


1.06 


1.5 


K~tt + p + n~ 


3.65 


0.00 


12 


11% 


15.4 


35.9 


K-TY+e+e' 


3.50 


0.00 


6 


15% 


7.53 


38.5 




5.25 


0.00 


15 


12% 


17.3 


55.3 




2.13 


0.17 





17% 


1.22 


3.3 


K+K'e+e- 


6.13 


0.04 


9 


18% 


9.61 


31.5 




3.50 


0.17 


5 


17% 


6.61 


18 


pW" 


0.00 


0.75 





10% 


1.80 


2.2 


p°e+e- 


0.00 


0.18 


1 


12% 


4.28 


12.4 




0.00 


0.82 


1 


11% 


3.60 


6.6 




0.30 


1.87 


3 


24% 


5.40 


2.4 


K*°e+e- 


0.88 


0.49 


2 


25% 


4.68 


4.7 




1.75 


2.30 


9 


24% 


12.8 


8.3 


(t>p + p~ 


0.30 


0.04 





33% 


2.33 


3.1 


4>e + e~ 


0.00 


0.01 





33% 


2.75 


5.9 




0.00 


0.05 





33% 


2.71 


4.7 


ir~ ir~ p + fi + 


0.91 


0.79 


1 


9% 


2.78 


2.9 


tv~ n~ e + e + 


0.00 


0.18 


1 


11% 


4.26 


11.2 


7T _ TT~ fl + e + 


2.63 


0.86 


4 


10% 


5.18 


7.9 


K~ir~n + fi + 


2.74 


3.96 


14 


9% 


15.7 


39.0 


K~tt~ e + e + 


0.88 


1.04 


2 


16% 


4.14 


20.6 


K^Tv'p + e+ 


0.00 


4.88 


7 


11% 


7.81 


21.8 


K~K~p, + p+ 


1.22 


0.00 


1 


17% 


3.27 


9.4 


K~K~e + e + 


0.88 


0.00 


2 


17% 


5.28 


15.2 


K-K-fi + e + 


0.00 


0.00 





17% 


2.52 


5.7 



The sources of systematic errors in this analysis in- 
clude: errors from the fit to the normalization sam- 
ple A^Norm! statistical uncertainty on the selection effi- 
ciencies, calculated for Monte Carlo simulated events, 
for both /notiii an d fx C 'i uncertainties in the calcula- 
tion of misidentification background; and uncertainties 
in the relative efficiency for each mode, including lep- 
ton tagging efficiencies. These tagging efficiency uncer- 
tainties include: 1) muon counter efficiencies from hard- 
ware performance; and 2) the fraction of signal events 
(based on simulations) that would remain outside the 
signal window due to bremsstrahlung tails. Also, for the 
D° — > p°£ + £ + modes, an additional systematic error is 
included because we are using D° — > 7r + 7r~7r + 7r~ as the 



normalization mode since there is no published branching 
fraction for D° — > p°7r + 7r~. The sums, taken in quadra- 
ture, of these systematic errors are listed in Table || 

In summary, we use a "blind" analysis of data from 
Fermilab experiment E791 to obtain upper limits on the 
dilepton branching fractions for 27 flavor-changing neu- 
tral current, lepton- number violating, and lepton- family 
violating decays of D° mesons. No evidence for any 
of these 3 and 4-body decays is found. Therefore, we 
present upper limits on the branching fractions at the 
90% confidence level. Four limits represent significant 
improvements over previously published results. Eigh- 
teen of these modes have no previously reported limits. 
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